EDIT: This is the intial test. Later wired into real CI, see bot comment below

POC: Automated spec audit using local LLM

I built a prototype that checks whether a PR's SPEC.md update matches its code changes, using qwen3:14b on Typhon via Ollama. It:

1. Fetches and splits the diff into code changes vs spec changes
2. Asks the LLM to summarize the code changes (without seeing the spec)
3. Asks the LLM to compare that summary to the spec diff

• The --worktree option now accepts a new value create:<branch-name> which creates a new git worktree and branch, then switches to bind mode in that worktree.
• Using --worktree=create without specifying a branch name now results in an error, requiring the format create:<branch-name>.
• When --worktree=create:<branch-name> is used, the tool creates a new worktree directory, initializes it with the specified branch, and changes the current working directory to the new worktree.
• The validation logic for --worktree values now includes the create:<branch-name> format in its regex pattern.
• The WORKTREE_MODE configuration variable now supports the create:<branch> value as a valid option.

Verdict: FAIL

Flag mismatch: The spec introduces a new flag (--create-worktree) instead of updating the existing --worktree flag. The code extends --worktree to accept create:<branch-name> as a value — the spec should reflect that, not add a separate flag.

Missing configuration: The spec adds a new config variable (WORKTREE_DIR) not present in the code changes.

Contradictory behavior: The spec's --create-worktree flag is not supported by the code, while the code's actual --worktree=create:<branch> behavior is not in the spec.

POC lives in https://github.com/asmacdo/typhon-ai-experiments (private) under exp-1-audit-spec-change/. Next steps: fix the spec on this PR to match, re-run to confirm it passes, then explore expanding that project so it can be used as a CI check.

Code change summary

Here's a summary of the new or changed functionality:

• New --worktree=create:<branch-name> option: Users can now specify --worktree=create:<branch-name> to automatically create a new Git worktree at a path derived from the current directory, switch to a new branch in that worktree, cd into it, and then run yolo with the bind mount behavior (--worktree=bind) applied to this newly created worktree.
• Error for bare --worktree=create: If --worktree=create is used without specifying a branch name (e.g., --worktree=create instead of --worktree=create:new-branch), the yolo script will now exit with an error, requiring a branch name to be provided.
• Dynamic container name for created worktrees: When --worktree=create:<branch-name> is used, the container name generated by yolo will now reflect the path of the newly created worktree, rather than the original directory where yolo was invoked.

Verdict

FAIL

Here's a breakdown of discrepancies and omissions:

1. Flag Name Contradiction:

   • Code Summary: States a new option --worktree=create:<branch-name>.
   • SPEC Diff: Introduces a new, separate flag named --create-worktree=BRANCH (and clarifies its syntax like --create-worktree=feature-x or --create-worktree=main:feature-x). The existing --worktree=MODE flag still only lists ask, bind, skip, error as valid modes, not create.
   • Problem: The user would interact with yolo using a different flag name than what the code summary implies. This is a direct contradiction in user-facing syntax.

2. Implicit --worktree=bind Behavior Not Documented:

   • Code Summary: "run yolo with the bind mount behavior (--worktree=bind) applied to this newly created worktree."
   • SPEC Diff (Section 5, Behavior point 4): "After cd, normal worktree detection and --worktree mode handling applies."
   • Problem: The code summary specifies that bind mounting is automatically applied. The spec, however, states "normal ... mode handling applies," which typically means the default ask or whatever is configured, not necessarily bind. If the bind behavior is truly automatic after --create-worktree, the spec is underselling or misrepresenting this important detail.

3. Container Naming Change Not Documented:

   • Code Summary: "When --worktree=create:<branch-name> is used, the container name generated by yolo will now reflect the path of the newly created worktree, rather than the original directory where yolo was invoked."
   • SPEC Diff: The provided SPEC.md diff does not include any changes to the "Container Naming" section (Section 6) or any other part that would describe this new behavior.
   • Problem: This is a clear omission of user-visible functionality. Users might expect the container name to be consistent with the original invocation path, but the code changes it, and the spec doesn't mention it.